In coatless/ITS: A Tour of Time Series Analysis with R

Linear Processes

The Backshift Operator

Definition

The Backshift Operator is helpful when manipulating time series. When we backshift, we are changing the indices of the time series. e.g. $t \rightarrow t-1$. The operator is defined as:

[B{X_t} = {X_{t - 1}}]

Backshifting on a constant, $c$, results in:

[Bc = c]

If we were to repeatedly apply the backshift operator, we would receive:

[\begin{aligned} {B^2}{X_t} &= B\left( {B{X_t}} \right) \ &= B\left( {{X_{t - 1}}} \right) \ &= {X_{t - 2}} \ \end{aligned}]

We can generalize this behavior as:

$${B^k}{X_t} = {X_{t - k}}$$

The backshift operator is helpful for later decompositions in addition to making differencing operations more straightforward.

The notation for the backshift operator changes depending on the author's preferences. An alternative way to denote a backshift operation given by $B$ is to use the lag operator $L$. The same conventions as discussed above apply equally to the lag operator.

Differencing Operator

Definition: Differencing Operator

The Differencing Operator is defined as the gradient symbol applied to a time series: [\nabla {X_t} = {X_t} - {X_{t - 1}}]

The differencing operator is helpful when trying to remove trend from the data. Many make an analogy between this operator and taking a first derivative of a function.

We can take higher moments of differences by: [\begin{aligned} {\nabla ^2}{X_t} &= \nabla \left( {\nabla {X_t}} \right) \ &= \nabla \left( {{X_t} - {X_{t - 1}}} \right) \ &= \left( {{X_t} - {X_{t - 1}}} \right) - \left( {{X_{t - 1}} - {X_{t - 2}}} \right) \ &= {X_t} - 2{X_{t - 1}} + {X_{t - 2}} \ \end{aligned} ]

So, the difference operator has the following properties: [\begin{aligned} {\nabla ^k}{X_t} &= {\nabla ^{k - 1}} \left( {\nabla {X_t}}\right) \ {\nabla ^1}{X_t} &= \nabla {X_t} \ \end{aligned} ]

Notice, within the difference operation, we are backshifting the timeseries.

If we rewrite the difference operator to use the backshift operator, we receive: [\nabla {X_t} = {X_t} - {X_{t - 1}} = \left( {1 - B} \right){X_t}]

This holds for later incarnations as well: [\begin{aligned} {\nabla ^2}{X_t} &= {X_t} - 2{X_{t - 1}} + {X_{t - 2}} \ &= \left( {1 - B} \right)\left( {1 - B} \right){X_t} \ &= {\left( {1 - B} \right)^2}{X_t} \ \end{aligned} ]

Thus, we can generalize this to: [{\nabla ^k}{X_t} = {\left( {1 - B} \right)^k}{X_t}]

Seasonal Differencing Operator

With a slight change to the difference operator, a seasonal difference can be applied by using a backshift greater than 1:

[{\nabla s}{X_t} = {X_t} - {X{t - s}} = \left( {1 - B_s} \right){X_t}]

In a similar vein, the above is able to be differenced by:

[{\nabla ^k_s}{X_t} = {\left( {1 - B_s} \right)^k}{X_t}]

Linear Process

Definition

A time series, $(X_t)$, is defined to be a linear process if it can be expressed as a linear combination of white noise by:

[{X_t} = \mu + \sum\limits_{j = - \infty }^\infty {{\psi j}{W{t - j}}} ]

where $W_t \sim WN(0, \sigma^2)$ and $\sum\limits_{j = - \infty }^\infty {\left| {{\psi j}} \right|} < \infty$. Note, the later assumption is required to ensure that the series has a limit. Furthermore, the set of coefficients [{{ {\psi _j}} {j = - \infty , \cdots ,\infty }}] can be viewed as linear filter. These coefficients do not have to be all equal nor symmetric as later examples will show. Generally, the properties of a linear process related to mean and variance are given by:

[\begin{aligned} \mu_{X} &= \mu \ \gamma_{X}(h) &= \sigma W^2\sum\limits{j = - \infty }^\infty {{\psi j}{\psi {h + j}}} \end{aligned}]

The latter is derived from

[\begin{aligned} \gamma \left( h \right) &= Cov\left( {{x_t},{x_{t + h}}} \right) \ &= Cov\left( {\mu + \sum\limits_{j = - \infty }^\infty {{\psi j}{w{t - j}}} ,\mu + \sum\limits_{j = - \infty }^\infty {{\psi j}{w{t + h - j}}} } \right) \ &= Cov\left( {\sum\limits_{j = - \infty }^\infty {{\psi j}{w{t - j}}} ,\sum\limits_{j = - \infty }^\infty {{\psi j}{w{t + h - j}}} } \right) \ &= \sum\limits_{j = - \infty }^\infty {{\psi j}{\psi {j + h}}Cov\left( {{w_{t - j}},{w_{t - j}}} \right)} \ &= \sigma w^2\sum\limits{j = - \infty }^\infty {{\psi j}{\psi {j + h}}} \ \end{aligned} ]

Within the above derivation, the key is to realize that $Cov\left( {{w_{t - j}},{w_{t + h - j}}} \right) = 0$ if $t - j \ne t + h - j$.

Lastly, one of the better formalities of a linear process is that it is able to be represented in a simplified form under the backshift operator:

[{X_t} = \psi \left( B \right){W_t}]

Example: White Noise

The white noise process $\left{X_t\right}$, defined in \@ref(wn), can be expressed as a linear process under:

[\psi _j = \begin{cases} 1 , &\mbox{ if } j = 0\ 0 , &\mbox{ if } |j| \ge 1 \end{cases}.]

and $\mu = 0$.

Therefore, $\left{X_t\right} \sim WN(0, \sigma^2_W)$

Example: Moving Average Order 1

Similarly, consider $\left{X_t\right}$ to be a MA(1) process, given by \@ref(ma1). The process can be expressed linearly under:

[\psi _j = \begin{cases} 1, &\mbox{ if } j = 0\ \theta , &\mbox{ if } j = 1 \ 0, &\mbox{ if } j \ge 2 \end{cases}.]

and $\mu = 0$.

Thus, we have: $X_t = W_t + \theta W_{t-1}$

Example: Symmetric Moving Average

Consider a symmetric moving average given by:

[{X_t} = \frac{1}{{2q + 1}}\sum\limits_{j = - q}^q {{W_{t + j}}} ]

Thus, $\left{X_t\right}$ is defined for $q + 1 \le t \le n-q$. The above process would be a linear process since:

[\psi _j = \begin{cases} \frac{1}{{2q + 1}} , &\mbox{ if } -q \le j \le q\ 0 , &\mbox{ if } |j| > q \end{cases}.]

and $\mu = 0$.

In practice, if $q = 1$, we would have:

[{X_t} = \frac{1}{3}\left( {{W_{t - 1}} + {W_t} + {W_{t + 1}}} \right)]

Example: Autoregressive Process of Order 1

A more intensive application of a linear process is $\left{X_t\right}$ as an AR1 process, defined in \@ref(ar1). The intensity comes from the necessity to define the weights with respect to the time lag.

[\psi _j = \begin{cases} \phi^j , &\mbox{ if } j \ge 0\ 0 , &\mbox{ if } j < 0 \end{cases}.]

and $\mu = 0$.

Under the condition that $\left| \phi \right| < 1$ the process can be considered to be the traditional $X_t = \phi X_{t-1} + W_t$. Furthermore, the process can also be considered to be an MA($\infty$)!

coatless/ITS documentation built on May 13, 2019, 8:45 p.m.